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U.  S.  ARMY  TEST  AND  EVALUATION  COMMAND 
BACKGROUND  DOCUMENT 


STATISTICAL  METHODS  OF  RELIABILITY  DETERMINATION 
1.  INTRODUCTION 

The  increased  emphasis  placed  upon  high  reliability  for  Army  missile 
and  rocket  systems  and  the  need  for  better  collection,  analysis,  and  reporting 
of  success  and  failure  data  has  made  the  measurement  of  reliability  that  is 
obtained  throughout  the  Engineering  Test  Program  one  of  the  most  important  in¬ 
dices  used  to  assess  missile  system  capability.  Reliability  and  its  companion 
index,  accuracy,  determined  under  various  test  conditions  are  two  factors  which 
are  relevant  in  any  missile  system  evaluation.  The  fundamental  purpose  of  re¬ 
liability  is  to  estimate  the  probability  of  the  successful  performance  of  an 
item  under  specified  conditions  and  to  compute  the  expected  value  of  this  prob¬ 
ability  with  a  specified  confidence  for  the  population  of  the  items  considered  . 

A  well  planned  and  executed  reliability  program  for  engineering  test 
results  in  data  that  reduces  to  provide: 

a.  Numerical  estimates  of  system  probability  of  successful  perfor¬ 
mance  under  specified  conditions. 

b.  The  confidence  or  degree  of  assurance  that  is  attained  in  the 
estimates  of  a.  above  as  measures  of  the  true  reliability. 

c.  The  existence  (or  non-existence)  of  significant  trends  in  reli¬ 
ability  as  a  function  of  test  conditions. 

d.  Failed  parts  information. 

e.  Reliability  estimat&s  that  are  part  of  the  factors  used  in  a 
system  effectiveness  evaluation. 


The  statistical  methods  of  reliability  used  in  planning,  collecting, 
analysis,  the  reporting  of  reliability  data  for  missile  and  rocket  systems 
during  the  Engineering  Test  program  are  discussed  in  the  following  sections. 

't-f 


RELIABILITY  REQUIREMENTS 


Missile  system  reliability  requirements  are  stated  in  the  following 

ways : 

a.  The  minimum  inflight  reliability  of  the  missile  shall  be  0.90 

against  a  specified  target  at  a  range  of  not  less  than  _  meters  and 

an  altitude  of  _____________  meters. 

b.  After  a  6  (or  12)  month  storage  the  missile  shall  undergo  a 
functional  checkout  with  a  reliability  of  at  least  0.95. 

c.  Missile  prefire  checkout  reliability  shall  be  at  least  0.95. 

d.  The  missile  system  availability1  shall  not  be  less  than  0.90. 

e.  The  tracking  (or  identification)  radar  shall  have  a  Mean  Time 

Between  Failure3  (MTBF)  of  not  less  than  _  hours.  r. 


See  “Glossary— 
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The  Mean  Time  Between  Failure  (MTBF)  is  sometimes  used  as  a  descriptor 
of  reliability  when  it  is  not  appropriate  to  express  the  reliability  in  terms  of 
a  probability. 

3.  TEST  OBJECTIVES  FOR  RELIABILITY 

In  any  laboratory  or  firing  test  to  be  conducted  upon  a  missile  sys¬ 
tem  or  subsystem  which  is  to  obtain,  among  other  information,  estimates  of  re¬ 
liability  the  following  questions  need  to  be  answered  in  the  planning  stage. 

a.  Under  what  specific  test  conditions  is  the  reliability  to  be 

measured? 

b.  What  probability  distributions  are  applicable?  Normal  (Gaussian)? 
Exponential?  Binomial?  etc. 

c.  Are  the  test  conditions  selected  fixed  for  this  particular  experi¬ 
ment,  or  are  they  randomly  chosen  from  a  larger  set  of  conditions? 

d.  What  are  the  chief  sources  of  variation  to  be  expected? 

e.  Hew  are  the  items  (components,  subsystem,  missiles),  to  be  allo¬ 
cated  among  the  test  conditions  (treatments)? 

f.  What  particular  statistical  methods  should  be  used  and  for  what 

purpose? 

g.  What  precautions  are  necessary  in  testing  so  that  the  data  are 
not  invalidated  for  statistical  analyses? 

h.  How  is  a  failure  defined?3 

i.  What  constitutes  a  no-test  category?4 

When  appropriate,  an  analysis  should  be  carried  out  with  dummy  data 
to  determine  if  the  results  are  answering  the  right  questions  and  if  a  more 
efficient  reliability  design  can  be  used  without  loss  of  essential  information. 

4.  COLLECTION  AND  FORMAT  OF  DATA  FOR  RELIABILITY  ANALYSES 

a.  Missile  reliability  from  preflight  checkout  and  inflight  perfor¬ 
mance  data.  Tables  I  through  V  can  be  used  for  scorekeeping  purposes  according 
to  the  kind  of  data  available.  The  entries  in  the  tables  are  the  number  of 
successes  (S)  and  failures  (F)  scored  for  the  various  major  assemblies  in  a 
missile  during  flight  (Table  I),  and  similarly  scored  for  missile  flight  per¬ 
formance  during  various  phases  of  the  trajectory  (Table  II). 

Table  III  gives  a  breakdown  of  one  definition  of  mission  reliability 
as  a  function  of  other  reliabilities.  Table  IV  is  a  combination  scorekeeping 
of  missile  performance  according  to  jamor  assemblies,  missile  flight  perfor¬ 
mance  in  various  phases,  and  impact  (intercept)  reliability. 


3  See  Glossary 

4  See  Glossary  for  possible  conditions  of  no-test 
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Table  I.  Missile  Performance 


Round 

Number 

Propulsion 

System 

Hydraulics 

Guidance 

Power 

Supply 

Structure 

1 

S 

S 

S 

S 

S 

2 

S 

F 

S 

S 

S 

3 

s 

S 

F 

F 

S 

Reli¬ 

ability 

Estimate 

s' 

S' 

1 

mm 

S' 

S'  +  F' 

S'  +  F' 

SSI 

S'  +  F' 

S'  *  No.  of  successes 

F'  =  No.  of  failures 

Table  II.  Flight  Performance 


Round  No  .. 

Launch -Boost 

Mid -Course 

Terminal 

1 

S 

S 

S 

2 

F 

S 

S 

3 

S 

S 

F 

Reliability 

S' 

S' 

S' 

Estimate 

S’  +  F' 

S'  +  F' 

S'  +  F' 

Table  III.  Mission  Reliability 


(Pre-Launch  Rel.)  x  (Launch  Rel.)  x  (Flight  Rel.) 
x  (Impact  or  Intercept  P*.l.)  x  (Warhead  Rel.) 

No.  of  Missiles  that  Pass  Pre-Launch  Checkout 
Total  No.  of  Missiles  Checked  Out 

No.  of  Missiles  that  Are  Successfully  Launched 
No.  of  Missiles  that  Pass  Pre-Launch  Checkout 

No.  of  Missiles  tj-jat  Experienced  No  Known  Subsystem 
Malfunction  Which  Would  Lead  to  an  Abort  or  a  Large 

Deviation  From  the  Flight  Path _ 

No.  of  Missiles  that  are  Successfully  Launched 

No.  of  Missiles  that  Attained  a  Miss  SL  a  Predetermined  Value 
No.  of  Missiles  that  Experienced  No  Known  Subsystem 
Malfunction  Which  Would  Lead  to  an  Abort  or  a  Large 
Deviation  From  the  Flight  Path 


Mission 

Reliability 

Pre -Launch 
Reliability 

Launch 

Reliability 

Flight 

Reliability 


Intercept 

Reliability 
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b.  Dependence  of  reliability  upon  test  conditions:  Table  V  illus¬ 
trates  the  format  for  collecting  the  data  used  in  determining  the  effect  of 
test  conditions  upon  missile  inflight  reliability.  The  number  of  successful 
(S)  and  failed  (F)  missiles  with  respect  to  inflight  reliability  are  tabulated 
according  to  a  two  way  classification  (here  two  ranges,  Rj,  and  0% ,  and  three 
altitudes,  Aj,  -  As ,  for  example). 

The  data  thus  collected  in  this  contingency  table  can  be  statistic¬ 
ally  analysed  to  determine  if  missile  inflight  reliability  is  significantly 
dependent  upon  these  test  conditions. 

c.  Reliability  Tests  on  Subsystems  or  Components: 

1)  Functional  checkouts 

a)  Data  logs  (either  discrete  or  continuous)  are  maintained 
which  are  complete  and  annotated.  The  environmental  con¬ 
ditions  under  which  the  test(s)  are  conducted  are  speci¬ 
fied,  and  any  necessary  changes  in  them  from  the  original 
plan  of  test  must  be  explicitly  noted. 

b)  Unusually  large  or  small  observations  of  the  variable(s) 
being  tested  should  be  carefully  noted.  If  specification 
limits  are  to  be  checked  against,  these  limits  (engineer¬ 
ing  or  statistical)  must  be  fully  described  and  understood 
as  to  their  meaning.6 

2)  Sensitivity  Tests 

a)  These  tests  are  of  the  go  -  no  go  type;  the  item  under 
test  either  operates  or  fails  to  operate  under  a  given 
input  or  stimulus.  The  statistical  analysis  of  the  re¬ 
sulting  data  depends  upon  the  objectives  of  the  plan  of 
test  and  how  the  data  are  to  be  collected. 

b)  The  Bruceton  or  up-and-down  method  is  one  of  the  statis¬ 
tical  techniques  most  often  used  in  analyzing  sensitivity 
data. 

3)  Life  Tests  on  Electronic  Items 

a)  For  electronic  items  which  are  to  be  life  tested  for  de¬ 
monstrating  whether  or  not  their  Mean  Time  Between  Failure 
equals  or  exceeds  specification  limits. 

b)  Electronic  equipment  undergoing  tests  to  determine  their 


5  Limits  of  error  or  uncertainty  allowed  are  often  vaguely  stated  which  leads 
to  a  misinterpretation  of  their  meaning.  See  Reference  A,  Section  4,  Chapter 
23  for  a  full  discussion  of  this  subject. 


-4- 


Table  IV.  Use  of  Flight  Data  In  Combined  Tables 


Table  IVa.  Numerical  Example 
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Table  V. 


Number  of  Successful 
At  Two  Ranges  and 


and  Failed  Missiles 
Three  Altitudes 


Target 

Altitude 

Target  Range 

Totals 

% 

K*  J 

S 

F 

s 

F 

S 

F 

AX 

As 

Aa 

TOTALS 

_ 

Figure  1  below  shews  an  example  of  the  format  of  the  data  that  are 
collected  for  testing  the  voltage  from  the  firing  point  at  which  some  squibs 
are  activated. 


Voltage  from 
Firing  Point 

SQUIB  NUMBER  0  -  no  fire 

X  -  fire 

1 

2 

3 

4 

5 

6 

7 

8 

9 

etc 

•  •  * 

•  •  • 

.  •  . 

0.6 

0.7 

0.8* 

0.9 

1.0 

1.1 

X 

0 

X 

X 

0 

X 

.  »  • 

X 

0 

_ 

•  •  • 

— 

X 

0 

*  Initial  voltage  at  which  approximately  one-half  of  the  squibs  will  be 
activated. 


Figure  1.  Fire,  No-Fire  Results  for  Squibs 


To  find  the  least  value  of  the  input  voltage  with  which  say,  0.99% 
of  the  aquibs  can  be  expected  to  fire  at  a  certain  confidence  level  is  one  of 
the  objectives  of  this  type  of  test.  Several  types  of  statistical  analyses 
are  used,  depending  upon  the  experimental  objectives  and  the  test  set-up. 
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reliability  and/or  availability  require  data  of  the 
following  type: 

(1)  Accurate  and  complete  time  to  failure  readings. 

(2)  The  total  number  of  failures  observed.6 

Table  VI  shows  the  format  to  be  used  for  collecting  further  data  cn 
this  type  of  test. 

Table  VI.  Number  of  Squibs  That  Fired  and  Didn't  Fire  at  Various 
Voltages  from  Firing  Point  and  Percent  of  Fired  Squibs 


Voltage  From 
Firing  Point 


Squibs  Fired 


Squibs  That 
Didn't  Fire 


Percent  of 
Fired  Squibs 


0.0 


0.5 

0.6 

0.7 

0.8 

0.9 

1.0 

1.1 


5. 

DESIGN  FOR  RELIABILITY 

5.1 

RELIABILITY  TEST  PLAN 

5.1.1 

Reliability  in  Engineering  Tests 

5. 1.1.1 

The  Types  of  Data  to  be  Collected 

a.  Attribute  or  discrete  data* 

1)  Number  of  go-no  go  (success  and  failure)  counts  in  the 
following  types  of  testing. 

a)  Ground  equipment  checkouts. 

b)  Missile  prefire  checkout. 

c)  Inflight  phase  of  missile. 

d)  Launch  phase  of  missile. 

e)  Sensitivity  tests  on  squibs,  S-i  devices;  drop  tests. 

f)  Reliability  (life)  tests  on  electronic  equipment.7 

6  A  failure  must  be  clearly  defined. 

7  Section  II  contains  more  information  on  the  statistics  of  life  testing. 
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In  all  the  tests  enumerated  above  a  concise  definition  of  a  success¬ 
ful  operation  or  mission  is  needed  for  the  item  under  test.  Also,  the  criteria 
must  be  determined  for  declaring  which  events  constitute  a  "no  test"  category. 

b.  Variable  or  continuous  data. 

1)  Time  readings. 

a)  Operating  times  on  items. 

b)  Down  times  due  to  maintenance  actions. 

c)  Arming  times  on  S&A  devices. 

d)  Measurements  of  time  as  an  independent  variable. 

e)  Ignition  and  burnout  times  of  propulsion  systems  under¬ 
going  specific  tests. 

2)  Linear  or  angular  measurements  (examples) . 

a)  Azimuth  and  pitch. 

b)  Arming  distance  of  S&A 

c)  Drifts  in  sensing  devices. 

3)  Other  physical  measurements. 

a)  Pounds  -  thrust. 

b)  Impulse. 

c)  Temperature. 

d)  Power,  volts,  etc. 

4)  Data  comprising  several  parameters  whose  effect  on  a  depen¬ 
dent  variable  is  considered  when  one  of  the  former  is  held 
fixed;  e.g.,  the  burst  point  of  a  warhead  on  a  missile  is  a 
function  of  position,  velocity,  fuzing,  etc.  What  is  the 
effect  of  a  constant  velocity  on  the  reliability  of  a  warhead 
burst  point? 

5.1. 1.2  Uses  of  Collected  Data  for  a  Reliability  Analysis 

a.  From  Attribute  data,  (go,  no-go  or  success-failure  type  of  data) . 

1)  Reliability  estimates  of  ground  checkout  equipment,  missile 
checkout.  Confidence  limits  on  the  true  reliability. 

2)  Reliability  estimates  of  launch  and  inflight  phases  of  the 
missile  with  corresponding  confidence  limits. 

3)  Reliability  and  safety  of  S&A  devices  as  a  function  of  dis¬ 
tance  from  armed  position. 

4)  Verify  with  a  prescribed  confidence  that  the  reliability  of 
an  item  for  a  given  time  is  at  least  a  specified  value. 

5)  Determine  if  missile  reliability  depends  significantly  upon 
test  conditions  (e.g.,  target  types,  target  range)  or  speci¬ 
fied  environmental  conditions  (e.g.,  missile  fired  after  a 
cold  temperature  treatment  and/or  shock  treatment) . 
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6)  Estimate  of  rion-system  failure  ratio,  i.e.,  the  proportion  of 
failures  that  cannot  be  attributed  to  any  system  malfunction. 
For  example,  the  holds  on  missile  firings  due  to  range  in¬ 
strumentation  delays,  range  safety,  etc. 

The  success/failure  data  collected  is  evaluated  probability  wise  by 
using  the  binomial  distribution  function  or  the  chi-square  statistic. 

b.  From  Variable  Data. 


1)  Estimates  of  a  parameter  which  depends  upon  one  or  more 
variables.  Regression  analysis  used. 

2)  Statistical  tolerance  limits  to  verify  that  performance 
limits  are  met  (or  reject  an  item  not  within  specifications). 

3)  Availability  and  maintainability  estimates  from  down  time 
and  Mean  Time  to  Failure  (MTTF)  data. 

4)  identification  of  large  biases  (or  real  effects)  in  random 
data. 

5.1.2  Reliability  Demonstrations  Tests.8 


5.1.2. 1  The  Types  of  Data  to  be  Collected. 


a.  The  number  of  failures  observed  on  the  item(s)  undergoing  test. 

b.  The  operational  life  of  the  item(s)  on  test. 

c.  The  time  at  which  the  test  is  to  be  terminated. 

d.  The  number  of  observed  failures  i *  lowed  on  the  test. 

e.  The  total  number  of  items  on  test. 


The 

life  test  and 


life  test  performed  under  condition  c.  is  called  a  truncated 
under  condition  d.,  a  censored  life  test.9 


5. 1.2. 2  The  Requirements  or  Specifications  in  a  Life  Test  May  be  of  the 
Following  Types: 


a.  The  minimum  acceptable  Mean  Time  Between  Failure  (MTBF)  with  the 
associated  confidence  level. 

b.  The  specified  MTBF  and  a  minimum  acceptable  MTBF  with  associated 
producer's  and  buyer's  risks.10  These  are  specifications  for  a  sequential  life 
test.11 

c.  The  specified  MTBF  (8_)  and  two  values  of  the  MTBF, 

0!  *  0.80  8q  and  6^  =  1.20  0^.  Also,  the  producer's  and  buyer's  risks.12 

fa  —  -  -  —  -  ,  i  * 

In  particular,  life  tests  conducted  in  laboratories  on  electronic  items. 

9  Reference  E  contains  a  full  discussion  on  the  statistics  used  in  planning 
and  evaluating  life  test  data. 

10  See  Glossary 

11  See  Glossary 

12  WSMR  Regulation  715-6,  Appendix  II,  contains  an  example  with  these  specifi¬ 
cations.  The  AMC  Reliability  Handbook,  AMCP  702-3,  October  1968,  also 
contains  examples  on  life  tests.  See  pages  F-44  through  F'56  for 
sequential  test  information. 
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It  is  assumed  that  the  time  to  failure  on  many  electronic  items 
undergoing  life  test  follows  an  exponential  probability  distribution.  13 
Reference  E  makes  this  assumption  in  all  of  the  statistical  methods.  There 
are  however,  times  to  failure  experienced  by  some  it:ras  on  test  that  follow 
different  probability  laws  (e.g.,  the  Weibull  distribution),  and  for  a  valid 
reliability  analysis  of  life  test  data  the  proper  distribution  functions  must 
be  ascertained  either  from  experience  files  or  a  suitable  analysis  of  the 
data.14 

6.  DATA  REDUCTION  AND  PRESENTATION 

6.1  FROM  ATTRIBUTE  (DISCRETE)  DATA 

a.  Reliability  of  Missile  During  Preflight  Checkout  Procedures 

If  missile  automatic  checkout  equipment  is  used,  and  X  is  the  number 
of  "go"  conditions  obtained  from  a  total  of  N  discrete  tests,  then  X/N  is  the 
reliability  estimate  of  the  missile  during  the  prefire  checkout.  The  data 
collected  for  this  purpose  must  be  unambiguous  and  complete  to  be  of  any  use. 
The  output  data  must  eigher  measure  the  "go"  or  "no  go"  condition  of  the  item 
under  test  and  not  be  compounded  with  other  effects. 

b.  Inflight  Reliability  of  the  Missile. 

The  inflight  reliability  estimate  of  the  missile  hinges  upon  the 
definition  of  a  successful  missile  during  this  phase  of  the  missile  trajectory. 
An  example  of  such  a  definition  is  given  in  Table  III,  (flight  reliability). 

In  all  such  cases  where  the  ratio  of  the  number  of  successes  to  the 
number  of  "trails"  is  computed,  we  have  a  "point"  estimate  of  the  reliability. 
How  good  this  point  estimate  is,  or  how  consistent  it  is  with  respect  to  the 
true  but  unknown  reliability,  is  discussed  in  paragraph  6.1.g.  (confidence 
limits  on  reliability) . 

Data  which  can  be  planned  for,  collected,  and  summarized  as  in  Tables 
I  -  IV  yield  these  point  estimates  of  reliability.  These  estimates  can  be  used 
separately  or  in  conjunction  with  each  other;  the  latter  use  will  require 
further  examination  and  is  discussed  in  paragraph  d  below. 

c.  Statistical  Analysis  for  Contingency  Tables. 

Tht  data  collected  on  missile  inflight  successes  and  failures  with 
respect  to  various  test  conditions  are  analyzed  with  regard  to  the  calculated 
expected  number  of  successes  and  failures.  A  comparison  is  made  between  these 
expected  values  and  the  actual  values  to  see  if  they  are  compatible  under  the 
assumption  that  missile  inflight  reliability  is  not  dependent  upon  the  test 


13 

14 


See  Glossary 

Reference  E  contains  several  statistical  tests  used  to  determine  if  the 
failure  times  are  exponentially  distributed. 
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conditions  stated.  The  Chi-aquare  (  s)  statistic  is  used  in  this  analysis.16 

d.  Reliability  as  the  Product  of  Subsystem  (or  Component). 
Reliabilities,  R  =  rj,  r2  .  .  .  r^ 

If  reliability  is  to  be  computed  as  the  product  of  tvo  or  more  com¬ 
ponent  reliabilities,  the  following  precautions  are  necessary. 

1)  The  components  must  be  statistically  Independent  of  each 
other,  i.e.,  the  failure  of  one  component  does  not  cause 
another  component  to  fail  and  the  components  are  not  sensi¬ 
tive  to  the  same  or  to  correlated  environmental  stresses.16 

2)  It  is  assumed  that  if  one  component  fails  the  system  falls 
(components  in  series  in  contrast  to  parallel). 

During  the  missile  system  engineering  test  and  evaluation  program, 
application  of  this  product  rule  for  reliability  is  not  always  valid,  and  when 
it  is  used  care  must  be  exercised  to  see  that  the  above  assumptions  are  reason¬ 
ably  satisfied. 

e.  Hit  Probability 
H  =*  AR, 

where  A  *  Accuracy  of  missile  at  target  measured  in  terms  of  the  probability 
of  the  missile  hitting  a  target  of  prescribed  size, 

and  R  *  Missile  inflight  reliability,  or  the  ratio  of  the  number  of  rounds 

that  hit  the  target  area  without  an  observed  malfunction  to  the 
number  of  successfully  launched  missiles. 

Such  a  probability  as  defined  above  is  a  conditional  probability, 
i.e.,  the  accuracy  is  determined  upon  the  condition  that  only  the  inflight 
successful  rounds  are  counted  in  the  accuracy  calculation.  It  will  be  noticed 
here  that  whereas  R  is  a  discrete  measurement,  A  is  not  (a  variable  or  continu¬ 
ous  measure).  Compare  with  the  definition  of  hit  reliability  (below  Table  IV). 

The  three  estimates  -  target  reliability,  hit  reliability,  and  hit 
probability  are  all  values  that  can  be  computed,  depending  upon  the  definition 
of  the  term  reliability  in  a  specific  program. 

f.  Graphical  Representation  of  the  Accumulative  Reliability 

1)  The  Accumulative  Reliability 

The  accumulative  missile  reliability  in  a  series  of  firings 
is  based  upon  the  accumulated  proportions  of  successes  to 


16  Reference  A,  Section  2,  Chapters  8  and  9. 
16  See  Reference  F,  pp.  1-6. 
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the  sum  of  successes  and  failures  at  each  time  of  firing. 
Table  VII  and  Figure  2  illustrate  the  method. 

2)  Moving  Average  Graph  of  Missile  Reliability 

The  equally  weighted  moving  average  reliability  graph  of 
N  points  (N  an  odd  integer)  is  based  upon  the  average  accumu 
lative  reliability  of  (a)  preceding  (Ell)  missiles ,  (b)  the 
'‘midpoint*’  missile,  and  (c)  the  succeeding  (N^_T)  missiles. 
This  average  or  smoothed  reliability  is  computed  at  the  mid¬ 
point,  Rq,  according  to  the  formula 


N-l 

2 


Ri 


If  the  resulting  values  are  plotted  upon  the  same  type  of  graph  as 
appears  in  Figure  2,  the  points  corresponding  to  the  accumulative  reliability 
of  the  first  (N-l)  rounds  and  the  last  (N-l)  rounds  are  excluded. 

2 

Depending  upon  the  total  number  of  rounds  to  be  considered,  N  may  be  5,  7,  9, 
or  larger. 


The  purpose  of  this  type  of  graph  is  to  smooth  out  the  large  short 
time  fluctuations  in  missile  reliability  and  identify  trends  (if  they  exist) 
in  reliability  over  longer  intervals  of  time. 

g.  Confidence  Limits  on  Reliability 

A  point  estimate  of  reliability  is  seldom  enough  information  about 
the  reliability  just  by  its  self.  Confidence  limits  provide  a  measure  of  how 
good  that  estimate  is  or  how  "reliable"  it  is.  Associated  with  a  confidence 
limit  is  the  confidence  level  or  confident  coefficient  which  is  the  degree  of 
assurance  made  about  the  confidence  limits. 

Thus  if  a  reliability  estimate  is 

R  —  0.50, 

the  true  reliability,  R,  is  at  least  0.50  with  387.. confidence,  i.e.,  the  fidu¬ 
cial  (confidence)  probability  that  the  true  reliability  is  at  least  0.50  is 
38%.  Stated  in  an  equation 

Prob  [  R  a  0.30  1  =  0.38 

Likewise,  Prob  [  R  ^  0.30  ]  =  0.85.  These  values,  0.50  and  0.30,  are  both 


17  See  Reference  G. 
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Table  VII.  Accumulative  Missile  Reliability 
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* 


lower  confidence  limits  on  reliability,  the  first  at  the  38%  confidence  level 
and  the  second  at  the  85%  level.  Upper  confidence  limits  can  be  given  as  well, 
although  the  lower  limits  are  more  appropriate  in  many  cases  (e.g.,  reliability 
requirements  are  most  often  stated  in  terms  of  minimum  values).18 

Confidence  limits  on  a  product  of  reliabilities  can  often  be  approxi¬ 
mated  with  sufficient  accuracy.  Whenever  this  can  be  done,  such  limits  are  an 
essential  part  of  the  reliability  analysis.19 

h.  Relationships  Between  Confidence  Limits  on  Reliability,  Confi¬ 
dence  Levels,  the  Number  of  Failures,  and  the  Sample  Size. 

There  are  three  methods  for  determining  any  one  of  these  factors  if 
the  others  are  knows  (discrete  data  assumed) : 

1)  By  published  tables.20 

2)  8y  graphs21 

3)  By  special  slide  rules.22 

This  information  is  used  to  construct  tables  of  predicted  reliabili¬ 
ties  assuming  different  sample  sizes,  confidence  levels,  and  the  number  of 
failures. 


6.2  FROM  CONTINUOUS  (VARIABLE  DATA) 

Reliability  estimation  (and  the  computation  of  confidence  limits) 
based  upon  continuous  data  involves  two  parameters  -  the  reliability  itself 
(or  availability)  and  the  Mean  Time  Between  Failure  (MTBF) .  The  fundamental 
formulas,  assuming  an  exponential  distribution  of  life-times ,23are 


where 


a.  Reliability  =  R 


probability  of  no  failure  in  time  t 


=e 


-  t/e 


0  -  MTBF 


b.  Availability  = 


MTBF 

MTBF  +  MDT 


where  MDT  =  Mean  Down  Time 

18  Lower  confidence  limits  on  the  reliability  thus  provide  a  means  of  deter¬ 
mining  whether  or  not  missile  reliability  requirements  have  been  met. 

13  One  such  procedure  is  outlined  in  8  below. 

2°  Reference  K. 

21  See  Reference  G. 

22 

A  confidence  Limit  Computer,  Sandia  Corp.  Reliability  Department,  Sandia 
Corp;  Albuquerque,  New  Mexico,  General  Dynamics  Corp.  Pomona,  California, 
also  has  prepared  a  useful  reliability  sliderule. 

23  If  the  life-times  are  Weibull  Distributed,  see  Reference  B,  pages  F-7 
through  F-10 
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Here,  availability  is  defined  as  the  probability  that  an  equipment 
will  be  operational  at  a  random  point  in  time.  The  MDT  may  or  may  not  include 
down  time  due  to  supply  and  administrative  delays. 

7.  RELIABILITY  AND  SAFETY 

7 . 1  PROBLEMS 

Reliability  enters  missile  safety  problems  in  three  ways: 

a.  Finding  the  probability  of  an  unsafe  condition  (e.g.,  the  event 
that  a  rocket  motor  will  prematurely  fire  because  of  improper  ignition  time, 
or  a  chamber  rupture  due  to  excessive  pressure) . 

b.  Finding  the  probability  of  a  surface-to-surface  missile  hitting 
within  a  designated  "friendly"  area. 

c.  To  determine  the  limits  of  a  safety  region  or  the  probability 
of  a  hit  within  such  a  region. 

7.2  DATA  COLLECTION 

Data  collected  are  of  the  following  types. 

a.  Ignition  times  of  rocket  motors  with  corresponding  other  data 
(e.g.,  thrust,  impulse,  chamber  pressure). 

b.  Boundary  values  of  the  friendly  area  and  its  centroid;  the 
dispersion  (variance)  values  of  the  missile  in  flight.  Geometry  of  the  zone 
of  approach  of  the  missile  to  the  area  in  question  and  the  nominal  missile 
impact  point. 

c.  Number  of  rounds  to  be  considered  and  their  coordinates  of  im¬ 
pact  (range,  deflection) .  The  geometrical  form  of  the  safety  region  and  the 
location  (within  or  outside  of  the  region)  of  prime  points  of  sub-areas  of 
i-'-orest. 

7.3  DATA  REDUCTION 

No  standard  set  of  formulas  can  be  given  that  can  be  applied  in 
general  to  every  case  mentioned  above.  Each  reliability  safety  problem  calls 
for  a  particular  mode  of  attack  in  its  solution.  The  probability  distributions 
associated  with  the  relevant  data  have  to  be  first  determined  (and  they  may 
not  be  known  in  their  entirety). 

A  decision  to  use  variable  or  discrete  data  in  the  analysis,  and  the 
sample  sizes  of  items  to  be  tested  for  safety  purposes  (with  the  corresponding 
high  probability  levels  of  no  failure)  are  part  of  the  plan  of  test  for  reli¬ 
ability  and  safety. 

If  the  safety  test  is  to  be  conducted  on  items  using  the  attributes 
criterion  (success  or  failure),  exhorbitantly  large  sample  sizes  are  required 
to  satisfy  very  low  probabilities  of  failure.  Testing  by  variables  utilizes 
much  smaller  sample  sizes  in  addition  to  providing  other  information,  e.  g. , 
the  effect  of  environmental  factors  upon  the  item  under  test. 
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8.  FORMULAS  FOR  FINDING  LOWER  CONFIDENCE  LIMIT  ON  A  PRODUCT  OF 

RELIABILITIES 

The  formula  for  n  (8.2)  gives  sufficiently  accurate  results  for  large 
sample  sizes.  It  is  based  upon  the  use  of  propagation  of  error  formula. 


■  7(E>V 


where  F  =  F  (Xi ,  Xg ,  . . . ,  Xfc) 


SYMBOLS  USED 


A  A  A 


Let  R  -  PI  P2  .  .  .  Pk 


where 


R  =  estimated  reliability. 


P,  -  £i 


(i  =  1,2,  ...,  k) 


‘i  n^  v  *  *  '  *  * »  ' 

■  the  ith  component  estimated  reliability, 

and  si  =  number  of  successes  in  ni  independent  tests. 

Find  the  lower  confidence  limit,  Rl>  on  the  true  reliability, 
R,  such  that 

Prob  [R  i  Rl  1  -  0.90 

where  0.90  is  the  prescribed  confidence  level. 

8.2  FORMULAS 


Compute ; 


y-  -y- 

L  Si  /  ni 


and  put  r  *  Rn 
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where  r  =  adjusted  value  to  be  used  in  Reference  I.  Interpolating  within  the 
tables  of  the  above  reference  (for  the  computed  values  of  n  and  r),  the  value  of 
R  is  obtained. 

L 

If  Reference  E  can  be  obtained,  Table  4(c),  p.  3.85,  is  then  utilized 
to  quickly  obtain  the  desired  value  of  R  .  In  using  this  table,  the  column 
headings  of  numbers  are,  for  this  particular  problem,  values  of  (n-r)  and  the 
row  heading  numbers  are  values  of  n. 


(EXAMPLE) 


Suppose 


R 


17.  20  IJL  3740 

18  x  21  x  15  "  5670 


0.66 


Then 


3.472  -  3  _  0,472 

0.1997  -  0.1698  "  0.0299 


15.8 


and  r  =  10.42. 

Using  Reference -1  with  (n-r)  »  5.4  and  n  -  15.8,  we  have  R  *  0.47, 
the  interpolated  value.  L 

9*  SSL  reliability  FORMULA  FOR  LIFE -TEST  DATA  THAT  FOLLOW  A  WEIBULL 

DISTRIBUTION 


It  is  assumed  that  the  sample  failure-times  have  been  satisfactorily 
checked  as  to  their  being  from  a  Weibull  population. 

9 . 1  SYMBOLS  USED 

Let  t  =*  the  failure-free  time,  or  mission  time,  of  the  equipment, 

T]  *  (eta),  the  scale  parameter  of  the  Weibull  probability  distri¬ 
bution,  and 

8  *  the  shape  parameter  of  this  distribution. 

9.2  FORMULA 

Reliability  ■  Prob  [no  failure  in  time  t] 


exp 


[-(*'*)’] 


0  /  1 


If  8  <  1,  the  failure  rate  is  relatively  high  in  early  life,  decreasing  with 
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passing  time.  If  g  >  1,  the  converse  is  true.  If  g  *  1,  the  Weibull  distri¬ 
bution  reduces  to  the  exponential  distribution  of  life-times,  g  *  1  corre¬ 
sponds  to  a  constant  failure  rate,  i.e.,  independent  of  time,  when  the  failures 
of  the  item(s)  under  test  occur  in  a  random  manner  -  after  the  debugging  period 
and  before  vearout  of  the  item. 

The  scale  and  shape  parameters ,  T]  and  g  respectively,  can  be  esti¬ 
mated  from  life-test  data  that  follow  the  Weibull  distribution.34 

Also,  tests  are  available  for  determining  whether  or  not  life-test 
data  obey  sufficiently  well  this  distribution.36 


34  See  Reference  J. 
36  See  Reference  B. 
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GLOSSARY 


1.  Availability: 

a.  Pointwise  availability  is  the  probability  that  an  item  will  be  opera¬ 
tional  at  any  given  time. 

b.  Interval  availability  is  the  proportion  of  time  during  which  an  item 
is  operational,  e.g.,  23  out  of  every  24  hours  for  a  search  radar. 

2.  Buyers  Risk;  The  chance  the  buyer  (or  consumer)  assumes  in  accepting  a 
substandard  product,  usually  designated  by  the  Greek  letter  0  and  numeri¬ 
cally  equal  to  a  probability.  In  the  statistics  of  life  testing,  p  is  the 
probability  of  accepting  an  item  with  a  MTBF  less  than  the  minimum  required 
value . 

3.  Confidence  Lim  t.  Lower;  The  least  value  that  the  true  reliability  (or 
MTBF)  can  assume  with  a  specified  degree  of  assurance,  assuming  that  relia¬ 
bility  (MTBF)  estimates  are  obtained  from  similar  items  in  repeated  tests 
under  the  same  conditions. 


Stated  in  the  form  of  an  equation,  if 


Prob  [  R  >  ]  =  P7«, 


then  R  = 

*L' 
.  and  P%  = 


true  but  unknown  reliability, 
the  P7.  lower  confidence  limit  on  R, 

the  confidence  level  (degree  of  assurance)  associated  with  the 
limit  or  bounds,  RT . 


P%  usually  has  the  value  of  95  or  90. 

m  f* /D 

4.  Exponential  Distribution:  A  probability  function  of  the  form  1  -  e  , 
which  gives  the  probability  that  an  item  will  fail  in  time  t  if  ©  is  the 
Mean  Time  Between  Failure.  (See  definition  of  MTBF.)  The  reliability 
function  is  e"*'®. 


5. 


Mean  Time  Between  Failure  (MTBF):  The  average  number  of  hours  of  life  per 
failure  calculated  from  the  equation 

MTBF  =  Total  Operating  Time 

Total  Number  of  Failures 


The  failure  rate  is  then  the  reciprocal  of  the  MTBF. 

6.  No-Test :  The  conditions  under  which  an  item  is  not  counted  in  reliability 
scorekeeping  procedures.  Possible  conditions  for  missile  system  tests  are: 
(1)  Drone  malfunction  after  missile  launch.  •  (2)  Missile  forced  beyond  its 
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design  limits.  (3)  Operator  errors  if  only  missile  hardware  capability  is 
being  scored.  (4)  Operations  which  are  not  completed  in  the  preflight 
period  to  the  desired  point  due  to  conditions  external  to  the  system  or 
where  no  data  exists  on  the  completions. 

7.  Producers  Risk:  The  chance  the  producer  assumes  in  rejecting  an  item  that 
meets  or  exceeds  stated  requirements,  usually  designated  by  the  Greek  letter 
a  and  numerically  equal  to  a  probability. 

8.  Reliability;  The  probability  that  an  item  will  successfully  perform  its 
intended  mission  under  stated  conditions.  Interval  reliability  is  defined 
as  the  product  of  pointwise  availability  and  reliability,  i.e., 

Prob  [  item  is  operational  at  any  time  t  ] 

x  Prob  [  item's  successful  performance  during  the 

time  interval  t,  -  t  ] 
i  o 

9.  Sequential  Life  Test;  A  test  which  proceeds  until  a  decision  is  reached  to 
either  accept  or  reject  the  item(s)  under  test  according  to  criteria  estab¬ 
lished  in  the  design  of  the  experiment.  Sequential  analysis  is  a  statisti¬ 
cal  technique  that  provides  the  criteria,  procedure,  and  mathematics  for 
this  type  of  test. 
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